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Abstract 

In this paper, we use the Poincare separation theorem for estimating the eigenvalues of the fine grid. We 
propose a randomized version of the algorithm where several different coarse grids are constructed thus 
leading to more comprehensive eigenvalue estimates. The proposed algorithm is suited for modern day 
multicore and distributed processing in the sense that no communication is required between the processors, 
however, at the cost of possible redundant computation. 



1 Introduction 

The problem of obtaining an approximation to eigenvalues and eigenvectors appears in several applications 
including data mining, chemical research, vibration analysis of mechanical structures, image processing etc. 
On the other hand, singular value decomposition has many useful applications in signal processing and 
statistics. For iterative methods, an estimate of extreme eigenvalue is useful for rapid Chebychev method [5] 
and in the construction of deflation preconditioners. An estimate of extreme eigenvalue leads to an estimate 
of condition number for symmetric positive matrix. 

Poincare separation theorem [3] states that the eigenvalues of coarse grid matrix P T AP are "sandwiched" 
between the eigenvalues of the fine grid matrix A. In this paper, we consider samples of randomized coarsen- 
ing scheme, i.e., the fine grid matrix is coarsened using special randomized interpolation operators P leading 
to several samples of coarse grids preferably with different distribution of eigenvalues. We then compute the 
eigenvalues of these coarse grid matrices. When a sufficiently large number of coarse grids are taken then 
the smallest eigenvalue (singular value) of the fine grid is approximated by the smallest of the eigenvalues 
(singular values) of the coarse grid matrices and the largest eigenvalue (singular value) of the fine grid is ap- 
proximated by the largest of the eigenvalues (singular values) of the coarse grid matrices. On the other hand, 
it is also possible to use the eigenvalues (singular values) of the coarse grid matrices as shifts for computing 
the eigenvalues(singular values) for the fine grid matrix. 

The proposed algorithm is well suited for modern day multi-core and multiprocessor era since coarsening and 
subsequently the eigenvalue (singular value) of the resulting coarse grid could be computed independently 
without performing any inter node communication. The only communication required is when we gather the 
eigenvalues (singular values) computed by the processors. Given that communication often becomes more 
costly relative to computation it is essential to degisn algorithms that minimize communication as much 
as possible even at the cost of small redundant computation. This is the main reason behind the method 
proposed in this paper. However, we do not show any results for parallel case and here we only focus our 
study in understanding the quality of our approach. 

The algorithms proposed has some similarity with the Jacobi-Davidson (JD) method [6] in the sense that 
both of these method try to approach the the eigenvalues of the fine grid via coarse grid, however, contrary 
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to the sophisticated Jacobi-Davidson method, the method proposed is based on brute force approach, i.e., 
the method relies on creating enough coarse grid samples such that one of these coarse grid leads to the 
desired eigenvalue or singular value. Moreover, unlike JD method where the matrix P keeps growing by one 
column during the outer iteration in our method P is fixed thus the coarse grid matrix P T AP is also fixed 
for each coarse grid sample. 

This paper is organized as follows. In section (2), we review essential theorems and motivation behind the 
algorithms proposed. In section (3), we explain steps from clustering to obtaining the coarse matrix. All the 
algorithms for computing the eigenvalues and eigenvectors are presented in section (4), here we also show 
some the results of some numerical experiments and finally section (5) concludes this paper. 



2 Poincare separation theorem 

Let Xi denote an arbitrary eigenvalue of A. The trace of an n x n matrix A is defined to be the sum of the 
elements on the main diagonal of A, i.e., 



tr(A)=J2< 



i=i 

. If f(x) = (x — \\) dl ■ ■ ■ (x — \k) dk is the characteristic polynomial of a matrix A, then tr(A) is defined as 
follows 

tr(A) = diAi + • • • + d k X k . 

We have the following relation 

n n 

= (i) 

i=l i=l 

Let K T denote the transpose of a matrix K and let I k denote the identity matrix of size k. Here we will see 
how poincare separates eigenvalues of two grids. 

Theorem 1 (Poincare). Let A be a symmetric n x n matrix with eigenvalues \\ < A2 < ■ • • < X n , and let P 
be a semi- orthogonal nx k matrix with the property that P T P — I k . The eigenvalues fii < H2 < ■ • ■ < Hn-k+i 
of P T AP are separated by the eigenvalues of A as follows 

K < Hi < A„_fe +i . (2) 

Proof. The theorem is proved in [4]. □ 

In Figure ([T]), we show a part of the spectrum where eigenvalues of a coarse grid is distributed among the 
fine grid eigenvalues. 

Theorem 2. If A is a real symmetric nx n matrix with eigenvalues Ai < A2 < • • • < X n , then the following 
holds 



min tr(P T AP) =J2\i, (3) 
' 1 

max tr{P T AP) =V A„_ fe+l . (4) 

PTp = I k 

i—1 

Proof. The theorem is proved in [3]. □ 
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Figure 1: Poincare separates for sky 10x10x10 zoomed 
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Theorem 3. If A is a real symmetric nxn matrix with eigenvalues Ai < A2 < 
conditions are satisfied 

1. tr(P T AP) is minimum and 

2. P T AP has simple eigenvalues 



< X n , and if the following 



the 



have 



Hi = A,, 1 < i < k, 
where fii < [12 < • • • < Mfe are eigenvalues of P T AP. 

Proof. Since P T AP has simple eigenvalues, we have tr(P T AP) = ^2i=ifM- Also, from Theorem [2] above, 
we have 



tr(P T AP) = J2\ 



(5) 



We shall prove the hypothesis by contradiction. From[T] we have ^ > A^. Let there exist j, 1 < j < k, such 
that fii > A^ then get 



tr(P T AP)=J2f H >^2\ i 

i=l i=l 

contradicting ([5]). Thus, we must have \ii — \. The proof is complete. 



(6) 



□ 



The theorem above tells us that if we are able to find a matrix P such that tr{P T AP) is minimum, then the 
first k smallest eigenvalues of the matrix A are simply the eigenvalues of the matrix P T AP provided P T AP 
has simple eigenvalues. 

Theorem 4. If A is a real symmetric nxn matrix with eigenvalues Ai < A2 < • • • < A n , and if the following 
conditions are satisfied 

1. tr(P T AP) is maximum and 

2. P T AP has simple eigenvalues 
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then we have 

Mi = K-k+i, l<i<k, 
where /ii < [i% < ••• < /ife are eigenvalues of P T AP. 

Proof. Since P T AP has simple eigenvalues, we have tr(P T AP) = Also, from Theorem [2] above, 

we have 

fc 

tr(P T AP) -^A Jl+l _ fc . (7) 

i=l 

We shall prove the hypothesis by contradiction. From[T] we have [ii < Xi. Let there exist j, 1 < j < k, such 
that fii < Xi, then we get 

k k 

tr(P T AP) = Y.^<Y. X * ( g ) 

i=l i=l 

contradicting Q. Thus, we must have fa — Xi. The proof is complete. □ 

The theorem above tells us that if we are able to find a matrix P such that tr(P T AP) is maximum, then the 
k largest eigenvalues of the matrix A are simply the eigenvalues of the matrix P 7 AP provided P T AP has 
simple eigenvalues. Determining first k smallest or k largest eigenvalues of a matrix is of prime importance 
in many applications. 

Theorem 5 (Poincare). Let A be a real m x n matrix with singular values 

a 1 {A) > a 2 (A) > ... 

and let U and V be two matrices of order m x p and nx q, respectively, such that U*U — I p and V*V = I q . 
Let B = U*AV with singular values 

<7 X {B) > a 2 (B) > ■ ■■ 
then the singular values of B are separated by the singular values of A as follows 

o~i{A) > o~i(B) > ai +r (A), i = 1, 2, • • ■ , min{m, n} 

where r — (m — p) + (n — q) 

Proof. The theorem is proved in [4]. □ 



3 Clustering to coarsening 

Our aim is to estimate the eigenvalues of the fine grid A via the eigenvalues of coarse grid (P T AP). Thus, the 
first step is clustering which then leads to the interpolation operator P as follows. First a set of aggregates 
Gi are defined. There are several different ways of doing aggregation (also described in [7]), some of them 
are as follows: 

• This approach is closely related to the classical AMG [3j where one first defines the set of nodes Si to 
which i is strongly negatively coupled, using the Strong/Weak coupling threshold /3: 

Si = {j \ a ij < -0 max|a ifc | }. 

Then an unmarked node i is chosen such that priority is given to the node with minimal M i; here Mi 
being the number of unmarked nodes that are strongly negatively coupled to i [3] . 
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• Several graph partitioning methods exists. Aggregation for AMG is created by calling a graph par- 
titioner with number of aggregates as an input. The subgraph being partitioned are considered as 
aggregates. For instance, in this paper we use this approach by giving a call to the METIS graph 
partitioner routine METIS _PartGraphKway with the graph of the matrix and number of partitions as 
input parameters. The partitioning information is obtained in the output argument "part" . The part 
array maps a given node to its partition, i.e., part(i) = j means that the node i is mapped to the jth 
partition. In fact, the part array essentially determines the interpolation operator P. For instance, we 
observe that the "part" array is a discrete many to one map. Thus, the ith aggregate Gi = part _1 (i), 
where 

part- 1 ^) ={je [1, N] | party) = i } 

• K-means clustering (see MATLAB): This clustering is defined in MATLAB and it produces random 
clustering i.e., a random "part" array defined above. 



Let J be the number of such aggregates, then the interpolation matrix P is defined as follows 

|l, if i e Gi 



Pr 



0. 



3 ' 

otherwise, 



(9) 



Here, l<i<N,l<j<J,N being the size of the original coefficient matrix A. Let N — 4 be the size of 
A. Let there be two aggregates, G\ — {1,3} and G2 — {2,4}, then the restriction operator P T is defined 

' 1 Further, we assume that the aggregates Gi are such that 



as follows P T 



10 1 



Gi n Gj = cj), for i ^ j and U* G t = [1, N] 



(10) 



Here [1, N] denotes the set of integers from 1 to N. Notice that the matrix P defined above is an N x J 
matrix but since it has only one non-zero entry (which are "one") per row, the matrix can be defined by a 
single array containing the indices of the non-zero entries. The coarse grid matrix A c may be computed as 
follows 

{A c )ij = ^2 ^2 a k i 

where 1 < i, j < N c , and ciki is the (A;, l)th entry of A. 



4 Randomized coarsening and its applications 

In this section, we list the algorithms that may lead to an approximation of eigenvalues or singular values. 
In algorithm |TJ), we show the steps for obtaining the eigenvalues of the input matrix A. Here, fi* denotes 
the jth eigenvalue of the ith coarse grid. Later in the algorithm at step (7) /u* is used as a shift to obtain the 
eigenvalue of the input matrix A. Since, Poincare separation theorem tells us that /z* will lie between two 
eigenvalues of the input matrix A, we expect it to converge to nearest one. However, it is possible that some 
other eigenvalue of other coarse grid also converges to the same eigenvalue and this redundant computation 
is inherent in this approach. In Algorithm similar algorithm related to singular values is shown. Notice 
here that two interpolation matrices namely U and V are needed. The procedure for obtaining them is same 
as for P except that we make use of two random clustering to construct the coarse grid matrix U*AVi. For 
clustering, we use of the "kmeans" clustering of MATLAB. 

In Algorithm ^ and Q, we present special cases of the algorithms presented in Algorithms ([!]) and ^ 
to compute extreme eigenvalues and singular values respectively. We simply extract only the largest and 
smallest eigenvalues of all coarse grids. In Figure we plot the singular values for rand(50) matrix available 
in MATLAB for 5 coarse grid samples. The coarse grid eigenvalues are then used as shift to determine the 
fine grid eigenvalues. In figure Efy, we see in detail how the shifts converge to the actual eigenvalues. 
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Algorithm 1 Eigenvalue estimate using multiple coarse grid 



l: INPUT: A, J, k 

2: OUTPUT: A = eigenvalues of A 

3: for i = 1 to J do 

4: A\ = PfAPi 

5: Extract /4, • ■ ■ n\} = eigenvalues(A* ) 

6: for j = 1 to k do 

7: A* = eigenvalues(A, fXj) // fxtj is the shift 
8: end for 

9: end for 

10: A = {A}, Ag, A3, . . . , A*} U {A?, A|, A§, . . . , A*} U • • • U {X{ , X J 2 , X{, . . . , X{} 



Algorithm 2 Singular value estimate using multiple coarse grid 
l: INPUT: A, J, k 

2: OUTPUT: £ =singular value estimates of A 
3: for i = 1 to J do 

4: Aj = U?ylV5 

5: Extract {a\,a\, <j\,... a l k } = eigenvalues(AJ.) 
6: for j = 1 to k do 

7: Sj(A)=singularvalue(y4, crj) // <rj is the shift 
8: end for 
9: end for 

10: S = {S{,Si,Si,...,4}U{S? ! I]2,S|,... ! S] 2 fc }U---U{Ef,^,E3^...,I]^ 



Figure 2: Poincare separates for rand(50), N c = 22, J=5 
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Algorithm 3 Extreme eigenvalues using multiple coarse grid 
l: INPUT: A, J 

2: OUTPUT: {h. max , A min } — approx. max and min eigenvalues of A 
3: for i = 1 to J do 

4: A* = PfAPi j j perform coarsening 
5; ^Inax — eigmax(^4*) // just find the largest eigenvalue 
6; lAnin — eigmin(A* ) / / just find the smallest eigenvalue 
7: end for 

8 ; A TOa2; — max{fl max , [i max , [t max , ■ • ■ , Mmaa;} 
9; A m i„ = min{fi min , [J. min , (l> m i n , ■ ■ ■ , Mmin} 



G 



Figure 3: Poincare separates for rand(50), iV c = 22, J=5 
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Algorithm 4 Extreme singular values using multiple coarse grid 
1: INPUT: A, J 

2: OUTPUT: {S mall E m i„ } = approx. max and min singular values of A 

3: for i = 1 to J do 

4: A* = U* AVi II perform coarsening 

5; ff mn = singularmax(A*) // just find the largest singular value 
6; a min = singularmin(v4* ) / / just find the smallest singular value 
7: end for 

8: ^max = rnax { a 'rnaxJ a maxi a maxi • ■ • ) a max} 
9 ; ^min min{o ' m j n , C m j n , CT m j n , • . . , f m j n } 
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